1 link tagged with all of: lakehouse + posthog + data warehouse
Click any tag below to further narrow down your results
Links
PostHog built its warehouse on DuckDB, then hit limits at scale and is now rebuilding it on a custom Trino engine with its own table format, HogLake. The post explains why DuckDB and DuckLake fell short for customers with very large, complex workloads.
- Single-node DuckDB was hard to run safely in the cloud because PostHog had to size each worker before queries arrived, and predicting CPU and memory per query was difficult.
- DuckLake's portable catalog schema limited optimizations: at PostHog's scale a single-table write could take over three minutes, and over 99% of catalog statistics rows belonged to deleted tables.
- At about 2 PB, roughly 1 trillion rows and 15.3 million snapshots, DuckLake broke in many ways, and PostHog ended up carrying patches in a fork instead of contributing upstream.